SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Language model</title>

</head>

<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Language model</span></span>

</h1>

</header>

</p><p>A <b>language model</b> is a <a href="Model#Conceptual_model" title="Model">model</a> of the human brain's ability to produce <a href="Natural_language" title="Natural language">natural language</a>.<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-2" class="reference"><a href="#cite_note-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup> Language models are useful for a variety of tasks, including <a href="Speech_recognition" title="Speech recognition">speech recognition</a>,<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup> <a href="Machine_translation" title="Machine translation">machine translation</a>,<sup id="cite_ref-Semantic_parsing_as_machine_translation_4-0" class="reference"><a href="#cite_note-Semantic_parsing_as_machine_translation-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> <a href="Natural_language_generation" title="Natural language generation">natural language generation</a> (generating more human-like text), <a href="Optical_character_recognition" title="Optical character recognition">optical character recognition</a>, <a href="Route_optimization" class="mw-redirect" title="Route optimization">route optimization</a>,<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup> <a href="Handwriting_recognition" title="Handwriting recognition">handwriting recognition</a>,<sup id="cite_ref-6" class="reference"><a href="#cite_note-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup> <a href="Grammar_induction" title="Grammar induction">grammar induction</a>,<sup id="cite_ref-7" class="reference"><a href="#cite_note-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup> and <a href="Information_retrieval" title="Information retrieval">information retrieval</a>.<sup id="cite_ref-ponte1998_8-0" class="reference"><a href="#cite_note-ponte1998-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-hiemstra1998_9-0" class="reference"><a href="#cite_note-hiemstra1998-9"><span class="cite-bracket">[</span>9<span class="cite-bracket">]</span></a></sup>

</p><p><a href="Large_language_model" title="Large language model">Large language models</a> (LLMs), currently their most advanced form, are predominantly based on <a href="Transformer_(machine_learning)" class="mw-redirect" title="Transformer (machine learning)">transformers</a> trained on larger datasets (frequently using texts <a href="Web_scraping" title="Web scraping">scraped</a> from the public <a href="Internet" title="Internet">internet</a>). They have superseded <a href="Recurrent_neural_network" title="Recurrent neural network">recurrent neural network</a>-based models, which had previously superseded the purely statistical models, such as the <a href="Word_n-gram_language_model" title="Word n-gram language model">word <i>n</i>-gram language model</a>.

</p>

<div class="mw-heading mw-heading2"><h2 id="History">History</h2></div>

<p><a href="Noam_Chomsky" title="Noam Chomsky">Noam Chomsky</a> did pioneering work on language models in the 1950s by developing a theory of <a href="Formal_grammar" title="Formal grammar">formal grammars</a>.<sup id="cite_ref-10" class="reference"><a href="#cite_note-10"><span class="cite-bracket">[</span>10<span class="cite-bracket">]</span></a></sup>

</p><p>In 1980, statistical approaches were explored and found to be more useful for many purposes than rule-based formal grammars. Discrete representations like <a href="Word_n-gram_language_model" title="Word n-gram language model">word <i>n</i>-gram language models</a>, with probabilities for discrete combinations of words, made significant advances.

</p><p>In the 2000s, continuous representations for words, such as <a href="Word2vec" title="Word2vec">word embeddings</a>, began to replace discrete representations.<sup id="cite_ref-11" class="reference"><a href="#cite_note-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup> Typically, the representation is a <a href="Real_number" title="Real number">real-valued</a> vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning, and common relationships between pairs of words like plurality or gender.

</p>

<div class="mw-heading mw-heading2"><h2 id="Pure_statistical_models">Pure statistical models</h2></div>

<p>In 1980, the first significant statistical language model was proposed, and during the decade IBM performed ‘<a href="Claude_Shannon" title="Claude Shannon">Shannon</a>-style’ experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text.<sup id="cite_ref-12" class="reference"><a href="#cite_note-12"><span class="cite-bracket">[</span>12<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading3"><h3 id="Models_based_on_word_n-grams">Models based on word <i>n</i>-grams</h3></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .excerpt-hat .mw-editsection-like{font-style:normal}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1236090951">

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable dablink excerpt-hat selfref">This section is an excerpt from <a href="Word_n-gram_language_model" title="Word n-gram language model">Word n-gram language model</a>.<span class="mw-editsection-like "><span class="mw-editsection-bracket">[</span><a class="external text external" href="https://en.wikipedia.org/w/index.php?title=Word_n-gram_language_model&action=edit">edit</a><span class="mw-editsection-bracket">]</span></span></div><div class="excerpt">

<p>A <a href="Word_n-gram_language_model" title="Word n-gram language model">word <i>n</i>-gram language model</a> is a purely statistical model of language. It has been superseded by <a href="Recurrent_neural_network" title="Recurrent neural network">recurrent neural network</a>–based models, which have been superseded by <a href="Large_language_model" title="Large language model">large language models</a>.<sup id="cite_ref-13" class="reference"><a href="#cite_note-13"><span class="cite-bracket">[</span>13<span class="cite-bracket">]</span></a></sup> It is based on an assumption that the probability of the next word in a sequence depends only on a fixed size window of previous words. If only one previous word is considered, it is called a bigram model; if two words, a trigram model; if <i>n</i> − 1 words, an <i>n</i>-gram model.<sup id="cite_ref-Word_n-gram_language_model_jm_14-0" class="reference"><a href="#cite_note-Word_n-gram_language_model_jm-14"><span class="cite-bracket">[</span>14<span class="cite-bracket">]</span></a></sup> Special tokens are introduced to denote the start and end of a sentence <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \langle s\rangle }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \langle s\rangle }</annotation>

</semantics>

</math></span><img src="./a3c56a2b45b7686b9b2602a2fc70934e60fb9d72.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:2.9ex; height:2.843ex;" alt="{\displaystyle \langle s\rangle }" loading="lazy"></span> and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \langle /s\rangle }">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \langle /s\rangle }</annotation>

</semantics>

</math></span><img src="./22589bb236f4585ebac14299e749f9cdb30a382b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.062ex; height:2.843ex;" alt="{\displaystyle \langle /s\rangle }" loading="lazy"></span>.

</p>

To prevent a zero probability being assigned to unseen words, each word's probability is slightly higher than its frequency count in a <a href="Corpus" class="mw-disambig" title="Corpus">corpus</a>. To calculate it, various methods were used, from simple "add-one" smoothing (assign a count of 1 to unseen <i>n</i>-grams, as an <a href="Uninformative_prior" class="mw-redirect" title="Uninformative prior">uninformative prior</a>) to more sophisticated models, such as <a href="Good%E2%80%93Turing_discounting" class="mw-redirect" title="Good–Turing discounting">Good–Turing discounting</a> or <a href="Katz's_back-off_model" title="Katz's back-off model">back-off models</a>.</div></div>

<div class="mw-heading mw-heading3"><h3 id="Exponential">Exponential</h3></div>

<p><a href="Principle_of_maximum_entropy" title="Principle of maximum entropy">Maximum entropy</a> language models encode the relationship between a word and the <i>n</i>-gram history using feature functions. The equation is

</p><p><span class="mwe-math-element mwe-math-element-block"><span class="mwe-math-mathml-display mwe-math-mathml-a11y" style="display: none;"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(w_{m}\mid w_{1},\ldots ,w_{m-1})={\frac {1}{Z(w_{1},\ldots ,w_{m-1})}}\exp(a^{T}f(w_{1},\ldots ,w_{m}))}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<msup>

</mrow>

</msup>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(w_{m}\mid w_{1},\ldots ,w_{m-1})={\frac {1}{Z(w_{1},\ldots ,w_{m-1})}}\exp(a^{T}f(w_{1},\ldots ,w_{m}))}</annotation>

</semantics>

</math></span></span>

</p><p>where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(w_{1},\ldots ,w_{m-1})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(w_{1},\ldots ,w_{m-1})}</annotation>

</semantics>

</math></span><img src="./db345c7489ff361147f1363116ee351dc88b5719.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:16.826ex; height:2.843ex;" alt="{\displaystyle Z(w_{1},\ldots ,w_{m-1})}" loading="lazy"></span> is the <a href="Partition_function_(mathematics)" title="Partition function (mathematics)">partition function</a>, <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle a}</annotation>

</semantics>

</math></span><img src="./ffd2487510aa438433a2579450ab2b3d557e5edc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.23ex; height:1.676ex;" alt="{\displaystyle a}" loading="lazy"></span> is the parameter vector, and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f(w_{1},\ldots ,w_{m})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f(w_{1},\ldots ,w_{m})}</annotation>

</semantics>

</math></span><img src="./899a4aa9d383560f5fe3446a9ac9693c225bb184.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:14.324ex; height:2.843ex;" alt="{\displaystyle f(w_{1},\ldots ,w_{m})}" loading="lazy"></span> is the feature function. In the simplest case, the feature function is just an indicator of the presence of a certain <i>n</i>-gram. It is helpful to use a prior on <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle a}</annotation>

</semantics>

</math></span><img src="./ffd2487510aa438433a2579450ab2b3d557e5edc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.23ex; height:1.676ex;" alt="{\displaystyle a}" loading="lazy"></span> or some form of <a href="Regularization_(mathematics)" title="Regularization (mathematics)">regularization</a>.

</p><p>The log-bilinear model is another example of an exponential language model.

</p>

<div class="mw-heading mw-heading3"><h3 id="Skip-gram_model">Skip-gram model</h3></div>

<div class="excerpt-block"><div role="note" class="hatnote navigation-not-searchable dablink excerpt-hat selfref">This section is an excerpt from <a href="Word_n-gram_language_model#Skip-gram_language_model" title="Word n-gram language model">Word n-gram language model § Skip-gram language model</a>.<span class="mw-editsection-like "><span class="mw-editsection-bracket">[</span><a class="external text external" href="https://en.wikipedia.org/w/index.php?title=Word_n-gram_language_model&action=edit#Skip-gram_language_model">edit</a><span class="mw-editsection-bracket">]</span></span></div><div class="excerpt">

<p>Skip-gram language model is an attempt at overcoming the data sparsity problem that the preceding model (i.e. word <i>n</i>-gram language model) faced. Words represented in an embedding vector were not necessarily consecutive anymore, but could leave gaps that are <i>skipped</i> over (thus the name "skip-gram").<sup id="cite_ref-15" class="reference"><a href="#cite_note-15"><span class="cite-bracket">[</span>15<span class="cite-bracket">]</span></a></sup>

</p><p>Formally, a <span class="texhtml mvar" style="font-style:italic;">k</span>-skip-<span class="texhtml mvar" style="font-style:italic;">n</span>-gram is a length-<span class="texhtml mvar" style="font-style:italic;">n</span> subsequence where the components occur at distance at most <span class="texhtml mvar" style="font-style:italic;">k</span> from each other.

</p><p>For example, in the input text:

</p>

<dl><dd><i>the rain in Spain falls mainly on the plain</i></dd></dl>

<p>the set of 1-skip-2-grams includes all the bigrams (2-grams), and in addition the subsequences

</p>

<dl><dd><i>the in</i>, <i>rain Spain</i>, <i>in falls</i>, <i>Spain mainly</i>, <i>falls on</i>, <i>mainly the</i>, and <i>on plain</i>.</dd></dl>

<p>In skip-gram model, semantic relations between words are represented by <a href="Linear_combination" title="Linear combination">linear combinations</a>, capturing a form of <a href="Compositionality" class="mw-redirect" title="Compositionality">compositionality</a>. For example, in some such models, if <span class="texhtml mvar" style="font-style:italic;">v</span> is the function that maps a word <span class="texhtml mvar" style="font-style:italic;">w</span> to its <span class="texhtml mvar" style="font-style:italic;">n</span>-d vector representation, then

</p><p><span class="mwe-math-element mwe-math-element-block"><span class="mwe-math-mathml-display mwe-math-mathml-a11y" style="display: none;"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle v(\mathrm {king} )-v(\mathrm {male} )+v(\mathrm {female} )\approx v(\mathrm {queen} )}">

</mrow>

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle v(\mathrm {king} )-v(\mathrm {male} )+v(\mathrm {female} )\approx v(\mathrm {queen} )}</annotation>

</semantics>

</math></span></span>

</p>

where ≈ is made precise by stipulating that its right-hand side must be the <a href="Nearest_neighbor_search" title="Nearest neighbor search">nearest neighbor</a> of the value of the left-hand side.<sup id="cite_ref-Word_n-gram_language_model_mikolov_16-0" class="reference"><a href="#cite_note-Word_n-gram_language_model_mikolov-16"><span class="cite-bracket">[</span>16<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-Word_n-gram_language_model_compositionality_17-0" class="reference"><a href="#cite_note-Word_n-gram_language_model_compositionality-17"><span class="cite-bracket">[</span>17<span class="cite-bracket">]</span></a></sup></div></div>

<div class="mw-heading mw-heading2"><h2 id="Neural_models">Neural models</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Recurrent_neural_network">Recurrent neural network</h3></div>

<p>Continuous representations or <a href="Word_embedding" title="Word embedding">embeddings of words</a> are produced in <a href="Recurrent_neural_network" title="Recurrent neural network">recurrent neural network</a>-based language models (known also as <i>continuous space language models</i>).<sup id="cite_ref-18" class="reference"><a href="#cite_note-18"><span class="cite-bracket">[</span>18<span class="cite-bracket">]</span></a></sup> Such continuous space embeddings help to alleviate the <a href="Curse_of_dimensionality" title="Curse of dimensionality">curse of dimensionality</a>, which is the consequence of the number of possible sequences of words increasing <a href="Exponential_growth" title="Exponential growth">exponentially</a> with the size of the vocabulary, further causing a data sparsity problem. Neural networks avoid this problem by representing words as non-linear combinations of weights in a neural net.<sup id="cite_ref-bengio_19-0" class="reference"><a href="#cite_note-bengio-19"><span class="cite-bracket">[</span>19<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading3"><h3 id="Large_language_models">Large language models</h3></div>

<div class="excerpt-block"><div role="note" class="hatnote navigation-not-searchable dablink excerpt-hat selfref">This section is an excerpt from <a href="Large_language_model" title="Large language model">Large language model</a>.<span class="mw-editsection-like "><span class="mw-editsection-bracket">[</span><a class="external text external" href="https://en.wikipedia.org/w/index.php?title=Large_language_model&action=edit">edit</a><span class="mw-editsection-bracket">]</span></span></div><div class="excerpt">

/* start https://en.wikipedia.org/ */

.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1246091330">

/* start https://en.wikipedia.org/ */

.mw-parser-output .sidebar{width:22em;float:right;clear:right;margin:0.5em 0 1em 1em;background:var(--background-color-neutral-subtle,#f8f9fa);border:1px solid var(--border-color-base,#a2a9b1);padding:0.2em;text-align:center;line-height:1.4em;font-size:88%;border-collapse:collapse;display:table}body.skin-minerva .mw-parser-output .sidebar{display:table!important;float:right!important;margin:0.5em 0 1em 1em!important}.mw-parser-output .sidebar-subgroup{width:100%;margin:0;border-spacing:0}.mw-parser-output .sidebar-left{float:left;clear:left;margin:0.5em 1em 1em 0}.mw-parser-output .sidebar-none{float:none;clear:both;margin:0.5em 1em 1em 0}.mw-parser-output .sidebar-outer-title{padding:0 0.4em 0.2em;font-size:125%;line-height:1.2em;font-weight:bold}.mw-parser-output .sidebar-top-image{padding:0.4em}.mw-parser-output .sidebar-top-caption,.mw-parser-output .sidebar-pretitle-with-top-image,.mw-parser-output .sidebar-caption{padding:0.2em 0.4em 0;line-height:1.2em}.mw-parser-output .sidebar-pretitle{padding:0.4em 0.4em 0;line-height:1.2em}.mw-parser-output .sidebar-title,.mw-parser-output .sidebar-title-with-pretitle{padding:0.2em 0.8em;font-size:145%;line-height:1.2em}.mw-parser-output .sidebar-title-with-pretitle{padding:0.1em 0.4em}.mw-parser-output .sidebar-image{padding:0.2em 0.4em 0.4em}.mw-parser-output .sidebar-heading{padding:0.1em 0.4em}.mw-parser-output .sidebar-content{padding:0 0.5em 0.4em}.mw-parser-output .sidebar-content-with-subgroup{padding:0.1em 0.4em 0.2em}.mw-parser-output .sidebar-above,.mw-parser-output .sidebar-below{padding:0.3em 0.8em;font-weight:bold}.mw-parser-output .sidebar-collapse .sidebar-above,.mw-parser-output .sidebar-collapse .sidebar-below{border-top:1px solid #aaa;border-bottom:1px solid #aaa}.mw-parser-output .sidebar-navbar{text-align:right;font-size:115%;padding:0 0.4em 0.4em}.mw-parser-output .sidebar-list-title{padding:0 0.4em;text-align:left;font-weight:bold;line-height:1.6em;font-size:105%}.mw-parser-output .sidebar-list-title-c{padding:0 0.4em;text-align:center;margin:0 3.3em}@media(max-width:640px){body.mediawiki .mw-parser-output .sidebar{width:100%!important;clear:both;float:none!important;margin-left:0!important;margin-right:0!important}}body.skin--responsive .mw-parser-output .sidebar a>img{max-width:none!important}@media screen{html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-list-title,html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle{background:transparent!important}html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle a{color:var(--color-progressive)!important}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-list-title,html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle{background:transparent!important}html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle a{color:var(--color-progressive)!important}}@media print{body.ns-0 .mw-parser-output .sidebar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r886047488">

/* start https://en.wikipedia.org/ */

.mw-parser-output .nobold{font-weight:normal}

/* end https://en.wikipedia.org/ */

</style><table class="sidebar sidebar-collapse nomobile nowraplinks"><tbody><tr><td class="sidebar-pretitle">Part of a series on</td></tr><tr><th class="sidebar-title-with-pretitle"><a href="Machine_learning" title="Machine learning">Machine learning</a><br>and <a href="Data_mining" title="Data mining">data mining</a></th></tr><tr><td class="sidebar-content">

<ul><li><a href="Supervised_learning" title="Supervised learning">Supervised learning</a></li>

<li><a href="Unsupervised_learning" title="Unsupervised learning">Unsupervised learning</a></li>

<li><a href="Semi-supervised_learning" class="mw-redirect" title="Semi-supervised learning">Semi-supervised learning</a></li>

<li><a href="Self-supervised_learning" title="Self-supervised learning">Self-supervised learning</a></li>

<li><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a></li>

<li><a href="Meta-learning_(computer_science)" title="Meta-learning (computer science)">Meta-learning</a></li>

<li><a href="Online_machine_learning" title="Online machine learning">Online learning</a></li>

<li><a href="Batch_learning" class="mw-redirect" title="Batch learning">Batch learning</a></li>

<li><a href="Curriculum_learning" title="Curriculum learning">Curriculum learning</a></li>

<li><a href="Rule-based_machine_learning" title="Rule-based machine learning">Rule-based learning</a></li>

<li><a href="Neuro-symbolic_AI" title="Neuro-symbolic AI">Neuro-symbolic AI</a></li>

<li><a href="Neuromorphic_engineering" class="mw-redirect" title="Neuromorphic engineering">Neuromorphic engineering</a></li>

<li><a href="Quantum_machine_learning" title="Quantum machine learning">Quantum machine learning</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<ul><li><a href="Statistical_classification" title="Statistical classification">Classification</a></li>

<li><a href="Generative_model" title="Generative model">Generative modeling</a></li>

<li><a href="Regression_analysis" title="Regression analysis">Regression</a></li>

<li><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></li>

<li><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></li>

<li><a href="Density_estimation" title="Density estimation">Density estimation</a></li>

<li><a href="Anomaly_detection" title="Anomaly detection">Anomaly detection</a></li>

<li><a href="Data_cleaning" class="mw-redirect" title="Data cleaning">Data cleaning</a></li>

<li><a href="Automated_machine_learning" title="Automated machine learning">AutoML</a></li>

<li><a href="Association_rule_learning" title="Association rule learning">Association rules</a></li>

<li><a href="Semantic_analysis_(machine_learning)" title="Semantic analysis (machine learning)">Semantic analysis</a></li>

<li><a href="Structured_prediction" title="Structured prediction">Structured prediction</a></li>

<li><a href="Feature_engineering" title="Feature engineering">Feature engineering</a></li>

<li><a href="Feature_learning" title="Feature learning">Feature learning</a></li>

<li><a href="Learning_to_rank" title="Learning to rank">Learning to rank</a></li>

<li><a href="Grammar_induction" title="Grammar induction">Grammar induction</a></li>

<li><a href="Ontology_learning" title="Ontology learning">Ontology learning</a></li>

<li><a href="Multimodal_learning" title="Multimodal learning">Multimodal learning</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><div style="display: inline-block; line-height: 1.2em; padding: .1em 0;"><a href="Supervised_learning" title="Supervised learning">Supervised learning</a><br><span class="nobold"><span style="font-size: 85%;">(<b><a href="Statistical_classification" title="Statistical classification">classification</a></b> • <b><a href="Regression_analysis" title="Regression analysis">regression</a></b>)</span></span> </div></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Apprenticeship_learning" title="Apprenticeship learning">Apprenticeship learning</a></li>

<li><a href="Decision_tree_learning" title="Decision tree learning">Decision trees</a></li>

<li><a href="Ensemble_learning" title="Ensemble learning">Ensembles</a>

<ul><li><a href="Bootstrap_aggregating" title="Bootstrap aggregating">Bagging</a></li>

<li><a href="Boosting_(machine_learning)" title="Boosting (machine learning)">Boosting</a></li>

<li><a href="Random_forest" title="Random forest">Random forest</a></li></ul></li>

<li><a href="Linear_regression" title="Linear regression">Linear regression</a></li>

<li><a href="Naive_Bayes_classifier" title="Naive Bayes classifier">Naive Bayes</a></li>

<li><a href="Artificial_neural_network" class="mw-redirect" title="Artificial neural network">Artificial neural networks</a></li>

<li><a href="Logistic_regression" title="Logistic regression">Logistic regression</a></li>

<li><a href="Perceptron" title="Perceptron">Perceptron</a></li>

<li><a href="Relevance_vector_machine" title="Relevance vector machine">Relevance vector machine (RVM)</a></li>

<li><a href="Support_vector_machine" title="Support vector machine">Support vector machine (SVM)</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="BIRCH" title="BIRCH">BIRCH</a></li>

<li><a href="Hierarchical_clustering" title="Hierarchical clustering">Hierarchical</a></li>

<li><a href="K-means_clustering" title="K-means clustering"><i>k</i>-means</a></li>

<li><a href="Fuzzy_clustering" title="Fuzzy clustering">Fuzzy</a></li>

<li><a href="Expectation%E2%80%93maximization_algorithm" title="Expectation–maximization algorithm">Expectation–maximization (EM)</a></li>

<li><br><a href="DBSCAN" title="DBSCAN">DBSCAN</a></li>

<li><a href="OPTICS_algorithm" title="OPTICS algorithm">OPTICS</a></li>

<li><a href="Mean_shift" title="Mean shift">Mean shift</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Factor_analysis" title="Factor analysis">Factor analysis</a></li>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Structured_prediction" title="Structured prediction">Structured prediction</a></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Graphical_model" title="Graphical model">Graphical models</a>

<ul><li><a href="Bayesian_network" title="Bayesian network">Bayes net</a></li>

<li><a href="Conditional_random_field" title="Conditional random field">Conditional random field</a></li>

<li><a href="Hidden_Markov_model" title="Hidden Markov model">Hidden Markov</a></li></ul></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Anomaly_detection" title="Anomaly detection">Anomaly detection</a></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Random_sample_consensus" title="Random sample consensus">RANSAC</a></li>

<li><a href="Local_outlier_factor" title="Local outlier factor">Local outlier factor</a></li>

<li><a href="Isolation_forest" title="Isolation forest">Isolation forest</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Neural_network_(machine_learning)" title="Neural network (machine learning)">Neural networks</a></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Autoencoder" title="Autoencoder">Autoencoder</a></li>

<li><a href="Deep_learning" title="Deep learning">Deep learning</a></li>

<li><a href="Feedforward_neural_network" title="Feedforward neural network">Feedforward neural network</a></li>

<li><a href="Recurrent_neural_network" title="Recurrent neural network">Recurrent neural network</a>

<li><a href="Reservoir_computing" title="Reservoir computing">reservoir computing</a></li></ul></li>

<li><a href="Boltzmann_machine" title="Boltzmann machine">Boltzmann machine</a>

<ul><li><a href="Restricted_Boltzmann_machine" title="Restricted Boltzmann machine">Restricted</a></li></ul></li>

<li><a href="Diffusion_model" title="Diffusion model">Diffusion model</a></li>

<li><a href="Convolutional_neural_network" title="Convolutional neural network">Convolutional neural network</a>

<li><a href="LeNet" title="LeNet">LeNet</a></li>

<li><a href="AlexNet" title="AlexNet">AlexNet</a></li>

<li><a href="DeepDream" title="DeepDream">DeepDream</a></li></ul></li>

<li><a href="Neural_field" title="Neural field">Neural field</a>

<ul><li><a href="Neural_radiance_field" title="Neural radiance field">Neural radiance field</a></li>

<li><a href="Physics-informed_neural_networks" title="Physics-informed neural networks">Physics-informed neural networks</a></li></ul></li>

<li><a href="Transformer_(deep_learning_architecture)" title="Transformer (deep learning architecture)">Transformer</a>

<ul><li><a href="Vision_transformer" title="Vision transformer">Vision</a></li></ul></li>

<li><a href="Mamba_(deep_learning_architecture)" title="Mamba (deep learning architecture)">Mamba</a></li>

<li><a href="Spiking_neural_network" title="Spiking neural network">Spiking neural network</a></li>

<li><a href="Memtransistor" title="Memtransistor">Memtransistor</a></li>

<li><a href="Electrochemical_RAM" title="Electrochemical RAM">Electrochemical RAM</a> (ECRAM)</li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a></div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Q-learning" title="Q-learning">Q-learning</a></li>

<li><a href="Policy_gradient_method" title="Policy gradient method">Policy gradient</a></li>

<li><a href="State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action" title="State–action–reward–state–action">SARSA</a></li>

<li><a href="Temporal_difference_learning" title="Temporal difference learning">Temporal difference (TD)</a></li>

<li><a href="Multi-agent_reinforcement_learning" title="Multi-agent reinforcement learning">Multi-agent</a>

</tr><tr><td class="sidebar-content">

<ul><li><a href="Active_learning_(machine_learning)" title="Active learning (machine learning)">Active learning</a></li>

<li><a href="Crowdsourcing" title="Crowdsourcing">Crowdsourcing</a></li>

<li><a href="Human-in-the-loop" title="Human-in-the-loop">Human-in-the-loop</a></li>

<li><a href="Mechanistic_interpretability" title="Mechanistic interpretability">Mechanistic interpretability</a></li>

</tr><tr><td class="sidebar-content">

<ul><li><a href="Coefficient_of_determination" title="Coefficient of determination">Coefficient of determination</a></li>

<li><a href="Confusion_matrix" title="Confusion matrix">Confusion matrix</a></li>

<li><a href="Learning_curve_(machine_learning)" title="Learning curve (machine learning)">Learning curve</a></li>

<li><a href="Receiver_operating_characteristic" title="Receiver operating characteristic">ROC curve</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Mathematical foundations</div><div class="sidebar-list-content mw-collapsible-content hlist">

<ul><li><a href="Kernel_machines" class="mw-redirect" title="Kernel machines">Kernel machines</a></li>

<li><a href="Bias%E2%80%93variance_tradeoff" title="Bias–variance tradeoff">Bias–variance tradeoff</a></li>

<li><a href="Computational_learning_theory" title="Computational learning theory">Computational learning theory</a></li>

<li><a href="Empirical_risk_minimization" title="Empirical risk minimization">Empirical risk minimization</a></li>

<li><a href="Occam_learning" title="Occam learning">Occam learning</a></li>

<li><a href="Probably_approximately_correct_learning" title="Probably approximately correct learning">PAC learning</a></li>

<li><a href="Statistical_learning_theory" title="Statistical learning theory">Statistical learning</a></li>

<li><a href="Vapnik%E2%80%93Chervonenkis_theory" title="Vapnik–Chervonenkis theory">VC theory</a></li>

<li><a href="Topological_deep_learning" title="Topological deep learning">Topological deep learning</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-content">

<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Journals and conferences</div><div class="sidebar-list-content mw-collapsible-content hlist">

<li><a href="Conference_on_Neural_Information_Processing_Systems" title="Conference on Neural Information Processing Systems">NeurIPS</a></li>

<li><a href="International_Joint_Conference_on_Artificial_Intelligence" title="International Joint Conference on Artificial Intelligence">IJCAI</a></li>

</tr><tr><td class="sidebar-content">

<ul><li><a href="Glossary_of_artificial_intelligence" title="Glossary of artificial intelligence">Glossary of artificial intelligence</a></li>

<li><a href="List_of_datasets_for_machine-learning_research" title="List of datasets for machine-learning research">List of datasets for machine-learning research</a>

<ul><li><a href="List_of_datasets_in_computer_vision_and_image_processing" title="List of datasets in computer vision and image processing">List of datasets in computer vision and image processing</a></li></ul></li>

<li><a href="Outline_of_machine_learning" title="Outline of machine learning">Outline of machine learning</a></li></ul></div></div></td>

</tr><tr><td class="sidebar-navbar"><style data-mw-deduplicate="TemplateStyles:r1239400231">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style></td></tr></tbody></table>A <a href="Large_language_model" title="Large language model">large language model</a> (LLM) is a language model trained with <a href="Self-supervised_learning" title="Self-supervised learning">self-supervised</a> <a href="Machine_learning" title="Machine learning">machine learning</a> on a vast amount of text, designed for <a href="Natural_language_processing" title="Natural language processing">natural language processing</a> tasks, especially <a href="Natural_language_generation" title="Natural language generation">language generation</a>.

The largest and most capable LLMs are <a href="Generative_pre-trained_transformer" title="Generative pre-trained transformer">generative pretrained transformers</a> (GPTs), which are largely used in <a href="Generative_artificial_intelligence" title="Generative artificial intelligence">generative</a> <a href="Chatbot" title="Chatbot">chatbots</a> such as <a href="ChatGPT" title="ChatGPT">ChatGPT</a>, <a href="Gemini_(chatbot)" title="Gemini (chatbot)">Gemini</a> or <a href="Claude_(language_model)" title="Claude (language model)">Claude</a>. LLMs can be <a href="Fine-tuning_(deep_learning)" title="Fine-tuning (deep learning)">fine-tuned</a> for specific tasks or guided by <a href="Prompt_engineering" title="Prompt engineering">prompt engineering</a>.<sup id="cite_ref-Large_language_model_few-shot-learners2_20-0" class="reference"><a href="#cite_note-Large_language_model_few-shot-learners2-20"><span class="cite-bracket">[</span>20<span class="cite-bracket">]</span></a></sup> These models acquire <a href="Predictive_learning" title="Predictive learning">predictive power</a> regarding <a href="Syntax" title="Syntax">syntax</a>, <a href="Semantics" title="Semantics">semantics</a>, and <a href="Ontology_(information_science)" title="Ontology (information science)">ontologies</a><sup id="cite_ref-21" class="reference"><a href="#cite_note-21"><span class="cite-bracket">[</span>21<span class="cite-bracket">]</span></a></sup> inherent in human <a href="Text_corpus" title="Text corpus">language corpora</a>, but they also inherit inaccuracies and <a href="Algorithmic_bias" title="Algorithmic bias">biases</a> present in the <a href="Training%2C_validation%2C_and_test_data_sets" title="Training, validation, and test data sets">data</a> they are trained in.<sup id="cite_ref-Large_language_model_Manning-2022_22-0" class="reference"><a href="#cite_note-Large_language_model_Manning-2022-22"><span class="cite-bracket">[</span>22<span class="cite-bracket">]</span></a></sup></div></div>

<p>Although sometimes matching human performance, it is not clear whether they are plausible <a href="Cognitive_model" title="Cognitive model">cognitive models</a>. At least for recurrent neural networks, it has been shown that they sometimes learn patterns that humans do not, but fail to learn patterns that humans typically do.<sup id="cite_ref-23" class="reference"><a href="#cite_note-23"><span class="cite-bracket">[</span>23<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading2"><h2 id="Evaluation_and_benchmarks">Evaluation and benchmarks</h2></div>

<p>Evaluation of the quality of language models is mostly done by comparison to human created sample benchmarks created from typical language-oriented tasks. Other, less established, quality tests examine the intrinsic character of a language model or compare two such models. Since language models are typically intended to be dynamic and to learn from data they see, some proposed models investigate the rate of learning, e.g., through inspection of learning curves.<sup id="cite_ref-24" class="reference"><a href="#cite_note-24"><span class="cite-bracket">[</span>24<span class="cite-bracket">]</span></a></sup>

</p><p>Various data sets have been developed for use in evaluating language processing systems.<sup id="cite_ref-:0_25-0" class="reference"><a href="#cite_note-:0-25"><span class="cite-bracket">[</span>25<span class="cite-bracket">]</span></a></sup> These include:

</p>

<ul><li><a href="Massive_Multitask_Language_Understanding" class="mw-redirect" title="Massive Multitask Language Understanding">Massive Multitask Language Understanding</a> (MMLU)<sup id="cite_ref-26" class="reference"><a href="#cite_note-26"><span class="cite-bracket">[</span>26<span class="cite-bracket">]</span></a></sup></li>

<li>Corpus of Linguistic Acceptability<sup id="cite_ref-27" class="reference"><a href="#cite_note-27"><span class="cite-bracket">[</span>27<span class="cite-bracket">]</span></a></sup></li>

<li>GLUE benchmark<sup id="cite_ref-28" class="reference"><a href="#cite_note-28"><span class="cite-bracket">[</span>28<span class="cite-bracket">]</span></a></sup></li>

<li>Microsoft Research Paraphrase Corpus<sup id="cite_ref-29" class="reference"><a href="#cite_note-29"><span class="cite-bracket">[</span>29<span class="cite-bracket">]</span></a></sup></li>

<li>Multi-Genre Natural Language Inference</li>

<li>Question Natural Language Inference</li>

<li>Quora Question Pairs<sup id="cite_ref-30" class="reference"><a href="#cite_note-30"><span class="cite-bracket">[</span>30<span class="cite-bracket">]</span></a></sup></li>

<li>Recognizing Textual Entailment<sup id="cite_ref-31" class="reference"><a href="#cite_note-31"><span class="cite-bracket">[</span>31<span class="cite-bracket">]</span></a></sup></li>

<li>Semantic Textual Similarity Benchmark</li>

<li>SQuAD question answering Test<sup id="cite_ref-32" class="reference"><a href="#cite_note-32"><span class="cite-bracket">[</span>32<span class="cite-bracket">]</span></a></sup></li>

<li>Stanford Sentiment <a href="Treebank" title="Treebank">Treebank</a><sup id="cite_ref-33" class="reference"><a href="#cite_note-33"><span class="cite-bracket">[</span>33<span class="cite-bracket">]</span></a></sup></li>

<li>Winograd NLI</li>

<li>BoolQ, PIQA, SIQA, HellaSwag, WinoGrande, ARC, OpenBookQA, NaturalQuestions, TriviaQA, RACE, BIG-bench hard, GSM8k, RealToxicityPrompts, WinoGender, CrowS-Pairs<sup id="cite_ref-34" class="reference"><a href="#cite_note-34"><span class="cite-bracket">[</span>34<span class="cite-bracket">]</span></a></sup></li></ul>

/* start https://en.wikipedia.org/ */

.mw-parser-output .portalbox{padding:0;margin:0.5em 0;display:table;box-sizing:border-box;max-width:175px;list-style:none}.mw-parser-output .portalborder{border:1px solid var(--border-color-base,#a2a9b1);padding:0.1em;background:var(--background-color-neutral-subtle,#f8f9fa)}.mw-parser-output .portalbox-entry{display:table-row;font-size:85%;line-height:110%;height:1.9em;font-style:italic;font-weight:bold}.mw-parser-output .portalbox-image{display:table-cell;padding:0.2em;vertical-align:middle;text-align:center}.mw-parser-output .portalbox-link{display:table-cell;padding:0.2em 0.2em 0.2em 0.3em;vertical-align:middle}@media(min-width:720px){.mw-parser-output .portalleft{margin:0.5em 1em 0.5em 0}.mw-parser-output .portalright{clear:right;float:right;margin:0.5em 0 0.5em 1em}}

/* end https://en.wikipedia.org/ */

</style>

/* start https://en.wikipedia.org/ */

.mw-parser-output .div-col{margin-top:0.3em;column-width:30em}.mw-parser-output .div-col-small{font-size:90%}.mw-parser-output .div-col-rules{column-rule:1px solid #aaa}.mw-parser-output .div-col dl,.mw-parser-output .div-col ol,.mw-parser-output .div-col ul{margin-top:0}.mw-parser-output .div-col li,.mw-parser-output .div-col dd{page-break-inside:avoid;break-inside:avoid-column}

/* end https://en.wikipedia.org/ */

</style><div class="div-col" style="column-width: 15em;">

<ul><li><a href="Artificial_intelligence_and_elections" title="Artificial intelligence and elections">Artificial intelligence and elections</a> – Use and impact of AI on political elections</li>

<li><a href="Cache_language_model" title="Cache language model">Cache language model</a></li>

<li><a href="Deep_linguistic_processing" title="Deep linguistic processing">Deep linguistic processing</a></li>

<li><a href="Ethics_of_artificial_intelligence" title="Ethics of artificial intelligence">Ethics of artificial intelligence</a></li>

<li><a href="Factored_language_model" title="Factored language model">Factored language model</a></li>

<li><a href="Generative_pre-trained_transformer" title="Generative pre-trained transformer">Generative pre-trained transformer</a></li>

<li><a href="Katz's_back-off_model" title="Katz's back-off model">Katz's back-off model</a></li>

<li><a href="Language_technology" title="Language technology">Language technology</a></li>

<li><a href="Semantic_similarity_network" title="Semantic similarity network">Semantic similarity network</a></li>

<li><a href="Statistical_model" title="Statistical model">Statistical model</a></li></ul>

</div>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFBlank2023" class="citation journal cs1">Blank, Idan A. (November 2023). <a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.tics.2023.08.006">"What are large language models supposed to model?"</a>. <i>Trends in Cognitive Sciences</i>. <b>27</b> (11): <span class="nowrap">987–</span>989. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.tics.2023.08.006">10.1016/j.tics.2023.08.006</a></span>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/37659920">37659920</a>.</cite>"LLMs are supposed to model how utterances behave." </span>

</li>

<li id="cite_note-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-2">^</a></b></span> <span class="reference-text"><cite id="CITEREFJurafskyMartin2021" class="citation book cs1">Jurafsky, Dan; Martin, James H. (2021). <a rel="nofollow" class="external text" href="https://web.stanford.edu/~jurafsky/slp3/3.pdf">"N-gram Language Models"</a> <span class="cs1-format">(PDF)</span>. <a rel="nofollow" class="external text" href="https://web.stanford.edu/~jurafsky/slp3/"><i>Speech and Language Processing</i></a> (3rd ed.). <a rel="nofollow" class="external text" href="https://web.archive.org/web/20220522005855/https://web.stanford.edu/~jurafsky/slp3/">Archived</a> from the original on 22 May 2022<span class="reference-accessdate">. Retrieved <span class="nowrap">24 May</span> 2022</span>.</cite></span>

</li>

<li id="cite_note-3"><span class="mw-cite-backlink"><b><a href="#cite_ref-3">^</a></b></span> <span class="reference-text">Kuhn, Roland, and Renato De Mori (1990). <a rel="nofollow" class="external text" href="https://www.researchgate.net/profile/Roland_Kuhn2/publication/3191800_Cache-based_natural_language_model_for_speech_recognition/links/004635184ee5b2c24f000000.pdf">"A cache-based natural language model for speech recognition"</a>. <i>IEEE transactions on pattern analysis and machine intelligence</i> 12.6: 570–583.</span>

</li>

<li id="cite_note-Semantic_parsing_as_machine_translation-4"><span class="mw-cite-backlink"><b><a href="#cite_ref-Semantic_parsing_as_machine_translation_4-0">^</a></b></span> <span class="reference-text">Andreas, Jacob, Andreas Vlachos, and Stephen Clark (2013). <a rel="nofollow" class="external text" href="https://www.aclweb.org/anthology/P13-2009">"Semantic parsing as machine translation"</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20200815080932/https://www.aclweb.org/anthology/P13-2009/">Archived</a> 15 August 2020 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).</span>

</li>

<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text"><cite id="CITEREFLiuWuLiuWang2023" class="citation journal cs1">Liu, Yang; Wu, Fanyou; Liu, Zhiyuan; Wang, Kai; Wang, Feiyue; Qu, Xiaobo (2023). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10587631">"Can language models be used for real-world urban-delivery route optimization?"</a>. <i>The Innovation</i>. <b>4</b> (6): 100520. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2023Innov...400520L">2023Innov...400520L</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.xinn.2023.100520">10.1016/j.xinn.2023.100520</a></span>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10587631">10587631</a></span>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/37869471">37869471</a>.</cite></span>

</li>

<li id="cite_note-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-6">^</a></b></span> <span class="reference-text">Pham, Vu, et al (2014). <a rel="nofollow" class="external text" href="https://arxiv.org/abs/1312.4569">"Dropout improves recurrent neural networks for handwriting recognition"</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201111170554/https://arxiv.org/abs/1312.4569">Archived</a> 11 November 2020 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>. 14th International Conference on Frontiers in Handwriting Recognition. IEEE.</span>

</li>

<li id="cite_note-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-7">^</a></b></span> <span class="reference-text">Htut, Phu Mon, Kyunghyun Cho, and Samuel R. Bowman (2018). <a rel="nofollow" class="external text" href="https://arxiv.org/pdf/1808.10000.pdf?source=post_page---------------------------">"Grammar induction with neural language models: An unusual replication"</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20220814010528/https://arxiv.org/pdf/1808.10000.pdf?source=post_page---------------------------">Archived</a> 14 August 2022 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1808.10000">1808.10000</a></span>.</span>

</li>

<li id="cite_note-ponte1998-8"><span class="mw-cite-backlink"><b><a href="#cite_ref-ponte1998_8-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFPonteCroft1998" class="citation conference cs1">Ponte, Jay M.; Croft, W. Bruce (1998). <i>A language modeling approach to information retrieval</i>. Proceedings of the 21st ACM SIGIR Conference. Melbourne, Australia: ACM. pp. <span class="nowrap">275–</span>281. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F290941.291008">10.1145/290941.291008</a>.</cite></span>

</li>

<li id="cite_note-hiemstra1998-9"><span class="mw-cite-backlink"><b><a href="#cite_ref-hiemstra1998_9-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFHiemstra1998" class="citation conference cs1">Hiemstra, Djoerd (1998). <i>A linguistically motivated probabilistically model of information retrieval</i>. Proceedings of the 2nd European conference on Research and Advanced Technology for Digital Libraries. LNCS, Springer. pp. <span class="nowrap">569–</span>584. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F3-540-49653-X_34">10.1007/3-540-49653-X_34</a>.</cite></span>

</li>

<li id="cite_note-10"><span class="mw-cite-backlink"><b><a href="#cite_ref-10">^</a></b></span> <span class="reference-text"><cite id="CITEREFChomsky1956" class="citation journal cs1">Chomsky, N. (September 1956). <span class="id-lock-subscription" title="Paid subscription required"><a rel="nofollow" class="external text" href="https://ieeexplore.ieee.org/document/1056813">"Three models for the description of language"</a></span>. <i>IRE Transactions on Information Theory</i>. <b>2</b> (3): <span class="nowrap">113–</span>124. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTIT.1956.1056813">10.1109/TIT.1956.1056813</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/2168-2712">2168-2712</a>.</cite></span>

</li>

<li id="cite_note-11"><span class="mw-cite-backlink"><b><a href="#cite_ref-11">^</a></b></span> <span class="reference-text"><cite class="citation news cs1"><a rel="nofollow" class="external text" href="https://cs.brown.edu/news/2022/02/22/the-nature-of-life-the-nature-of-thinking-looking-back-on-eugene-charniaks-work-and-life/">"The Nature Of Life, The Nature Of Thinking: Looking Back On Eugene Charniak's Work And Life"</a>. 22 February 2022. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20241103134558/https://cs.brown.edu/news/2022/02/22/the-nature-of-life-the-nature-of-thinking-looking-back-on-eugene-charniaks-work-and-life/">Archived</a> from the original on 3 November 2024<span class="reference-accessdate">. Retrieved <span class="nowrap">5 February</span> 2025</span>.</cite></span>

</li>

<li id="cite_note-12"><span class="mw-cite-backlink"><b><a href="#cite_ref-12">^</a></b></span> <span class="reference-text"><cite id="CITEREFRosenfeld2000" class="citation journal cs1">Rosenfeld, Ronald (2000). <a rel="nofollow" class="external text" href="https://figshare.com/articles/journal_contribution/6611138">"Two decades of statistical language modeling: Where do we go from here?"</a>. <i>Proceedings of the IEEE</i>. <b>88</b> (8): <span class="nowrap">1270–</span>1278. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2F5.880083">10.1109/5.880083</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:10959945">10959945</a>.</cite></span>

</li>

<li id="cite_note-13"><span class="mw-cite-backlink"><b><a href="#cite_ref-13">^</a></b></span> <span class="reference-text"><cite id="CITEREFBengioDucharmeVincentJanvin2003" class="citation journal cs1">Bengio, Yoshua; Ducharme, Réjean; Vincent, Pascal; Janvin, Christian (1 March 2003). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/10.5555/944919.944966">"A neural probabilistic language model"</a>. <i>The Journal of Machine Learning Research</i>. <b>3</b>: <span class="nowrap">1137–</span>1155 – via ACM Digital Library.</cite></span>

</li>

<li id="cite_note-Word_n-gram_language_model_jm-14"><span class="mw-cite-backlink"><b><a href="#cite_ref-Word_n-gram_language_model_jm_14-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFJurafskyMartin2023" class="citation book cs1">Jurafsky, Dan; Martin, James H. (7 January 2023). "N-gram Language Models". <a rel="nofollow" class="external text" href="https://web.stanford.edu/~jurafsky/slp3/ed3book_jan72023.pdf"><i>Speech and Language Processing</i></a> <span class="cs1-format">(PDF)</span> (3rd edition draft ed.)<span class="reference-accessdate">. Retrieved <span class="nowrap">24 May</span> 2022</span>.</cite></span>

</li>

<li id="cite_note-15"><span class="mw-cite-backlink"><b><a href="#cite_ref-15">^</a></b></span> <span class="reference-text"><cite id="CITEREFDavid_Guthrie2006" class="citation web cs1">David Guthrie; et al. (2006). <a rel="nofollow" class="external text" href="https://web.archive.org/web/20170517144625/http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf">"A Closer Look at Skip-gram Modelling"</a> <span class="cs1-format">(PDF)</span>. Archived from <a rel="nofollow" class="external text" href="http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf">the original</a> <span class="cs1-format">(PDF)</span> on 17 May 2017<span class="reference-accessdate">. Retrieved <span class="nowrap">27 April</span> 2014</span>.</cite></span>

</li>

<li id="cite_note-Word_n-gram_language_model_mikolov-16"><span class="mw-cite-backlink"><b><a href="#cite_ref-Word_n-gram_language_model_mikolov_16-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFMikolovChenCorradoDean2013" class="citation arxiv cs1">Mikolov, Tomas; Chen, Kai; Corrado, Greg; Dean, Jeffrey (2013). "Efficient estimation of word representations in vector space". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1301.3781">1301.3781</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>

</li>

<li id="cite_note-Word_n-gram_language_model_compositionality-17"><span class="mw-cite-backlink"><b><a href="#cite_ref-Word_n-gram_language_model_compositionality_17-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFMikolovSutskeverChenCorrado2013" class="citation conference cs1">Mikolov, Tomas; Sutskever, Ilya; Chen, Kai; Corrado, Greg S.; Dean, Jeff (2013). <a rel="nofollow" class="external text" href="http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf"><i>Distributed Representations of Words and Phrases and their Compositionality</i></a> <span class="cs1-format">(PDF)</span>. <a href="Advances_in_Neural_Information_Processing_Systems" class="mw-redirect" title="Advances in Neural Information Processing Systems">Advances in Neural Information Processing Systems</a>. pp. <span class="nowrap">3111–</span>3119. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201029083132/https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf">Archived</a> <span class="cs1-format">(PDF)</span> from the original on 29 October 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">22 June</span> 2015</span>.</cite></span>

</li>

<li id="cite_note-18"><span class="mw-cite-backlink"><b><a href="#cite_ref-18">^</a></b></span> <span class="reference-text"><cite id="CITEREFKarpathy" class="citation web cs1">Karpathy, Andrej. <a rel="nofollow" class="external text" href="https://karpathy.github.io/2015/05/21/rnn-effectiveness/">"The Unreasonable Effectiveness of Recurrent Neural Networks"</a>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201101215448/http://karpathy.github.io/2015/05/21/rnn-effectiveness/">Archived</a> from the original on 1 November 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">27 January</span> 2019</span>.</cite></span>

</li>

<li id="cite_note-bengio-19"><span class="mw-cite-backlink"><b><a href="#cite_ref-bengio_19-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFBengio2008" class="citation encyclopaedia cs1">Bengio, Yoshua (2008). <a rel="nofollow" class="external text" href="http://www.scholarpedia.org/article/Neural_net_language_models">"Neural net language models"</a>. <i><a href="Scholarpedia" title="Scholarpedia">Scholarpedia</a></i>. Vol. 3. p. 3881. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2008SchpJ...3.3881B">2008SchpJ...3.3881B</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.4249%2Fscholarpedia.3881">10.4249/scholarpedia.3881</a></span>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201026161505/http://www.scholarpedia.org/article/Neural_net_language_models">Archived</a> from the original on 26 October 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">28 August</span> 2015</span>.</cite></span>

</li>

<li id="cite_note-Large_language_model_few-shot-learners2-20"><span class="mw-cite-backlink"><b><a href="#cite_ref-Large_language_model_few-shot-learners2_20-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFBrownMannRyderSubbiah2020" class="citation journal cs1">Brown, Tom B.; Mann, Benjamin; Ryder, Nick; Subbiah, Melanie; Kaplan, Jared; Dhariwal, Prafulla; Neelakantan, Arvind; Shyam, Pranav; Sastry, Girish; Askell, Amanda; Agarwal, Sandhini; Herbert-Voss, Ariel; Krueger, Gretchen; Henighan, Tom; Child, Rewon; Ramesh, Aditya; Ziegler, Daniel M.; Wu, Jeffrey; Winter, Clemens; Hesse, Christopher; Chen, Mark; Sigler, Eric; Litwin, Mateusz; Gray, Scott; Chess, Benjamin; Clark, Jack; Berner, Christopher; McCandlish, Sam; Radford, Alec; Sutskever, Ilya; Amodei, Dario (December 2020). Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.F.; Lin, H. (eds.). <a rel="nofollow" class="external text" href="https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf">"Language Models are Few-Shot Learners"</a> <span class="cs1-format">(PDF)</span>. <i>Advances in Neural Information Processing Systems</i>. <b>33</b>. Curran Associates, Inc.: <span class="nowrap">1877–</span>1901. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/2005.14165">2005.14165</a></span>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20231117204007/https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf">Archived</a> <span class="cs1-format">(PDF)</span> from the original on 17 November 2023<span class="reference-accessdate">. Retrieved <span class="nowrap">14 March</span> 2023</span>.</cite></span>

</li>

<li id="cite_note-21"><span class="mw-cite-backlink"><b><a href="#cite_ref-21">^</a></b></span> <span class="reference-text"><cite id="CITEREFFathallahDasDe_GiorgisPoltronieri2024" class="citation conference cs1">Fathallah, Nadeen; Das, Arunav; De Giorgis, Stefano; Poltronieri, Andrea; Haase, Peter; Kovriguina, Liubov (26 May 2024). <a rel="nofollow" class="external text" href="https://2024.eswc-conferences.org/wp-content/uploads/2024/05/77770034.pdf"><i>NeOn-GPT: A Large Language Model-Powered Pipeline for Ontology Learning</i></a> <span class="cs1-format">(PDF)</span>. Extended Semantic Web Conference 2024. Hersonissos, Greece.</cite></span>

</li>

<li id="cite_note-Large_language_model_Manning-2022-22"><span class="mw-cite-backlink"><b><a href="#cite_ref-Large_language_model_Manning-2022_22-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFManning2022" class="citation journal cs1"><a href="Christopher_D._Manning" title="Christopher D. Manning">Manning, Christopher D.</a> (2022). <a rel="nofollow" class="external text" href="https://www.amacad.org/publication/human-language-understanding-reasoning">"Human Language Understanding & Reasoning"</a>. <i>Daedalus</i>. <b>151</b> (2): <span class="nowrap">127–</span>138. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1162%2Fdaed_a_01905">10.1162/daed_a_01905</a></span>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:248377870">248377870</a>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20231117205531/https://www.amacad.org/publication/human-language-understanding-reasoning">Archived</a> from the original on 17 November 2023<span class="reference-accessdate">. Retrieved <span class="nowrap">9 March</span> 2023</span>.</cite></span>

</li>

<li id="cite_note-23"><span class="mw-cite-backlink"><b><a href="#cite_ref-23">^</a></b></span> <span class="reference-text"><cite id="CITEREFHornsteinLasnikPatel-GroszYang2018" class="citation book cs1">Hornstein, Norbert; Lasnik, Howard; Patel-Grosz, Pritty; Yang, Charles (9 January 2018). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=XoxsDwAAQBAJ&dq=adger+%22goldilocks%22&pg=PA153"><i>Syntactic Structures after 60 Years: The Impact of the Chomskyan Revolution in Linguistics</i></a>. Walter de Gruyter GmbH & Co KG. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-5015-0692-5</bdi>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20230416160343/https://books.google.com/books?id=XoxsDwAAQBAJ&dq=adger+%22goldilocks%22&pg=PA153">Archived</a> from the original on 16 April 2023<span class="reference-accessdate">. Retrieved <span class="nowrap">11 December</span> 2021</span>.</cite></span>

</li>

<li id="cite_note-24"><span class="mw-cite-backlink"><b><a href="#cite_ref-24">^</a></b></span> <span class="reference-text"><cite id="CITEREFKarlgrenSchutze2015" class="citation cs2">Karlgren, Jussi; Schutze, Hinrich (2015), "Evaluating Learning Language Representations", <i>International Conference of the Cross-Language Evaluation Forum</i>, Lecture Notes in Computer Science, Springer International Publishing, pp. <span class="nowrap">254–</span>260, <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-3-319-64206-2_8">10.1007/978-3-319-64206-2_8</a>, <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9783319642055</bdi></cite></span>

</li>

<li id="cite_note-:0-25"><span class="mw-cite-backlink"><b><a href="#cite_ref-:0_25-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFDevlinChangLeeToutanova2018" class="citation arxiv cs1">Devlin, Jacob; Chang, Ming-Wei; Lee, Kenton; Toutanova, Kristina (10 October 2018). "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1810.04805">1810.04805</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>

</li>

<li id="cite_note-26"><span class="mw-cite-backlink"><b><a href="#cite_ref-26">^</a></b></span> <span class="reference-text"><cite id="CITEREFHendrycks2023" class="citation cs2">Hendrycks, Dan (14 March 2023), <a rel="nofollow" class="external text" href="https://github.com/hendrycks/test"><i>Measuring Massive Multitask Language Understanding</i></a>, <a rel="nofollow" class="external text" href="https://web.archive.org/web/20230315011614/https://github.com/hendrycks/test">archived</a> from the original on 15 March 2023<span class="reference-accessdate">, retrieved <span class="nowrap">15 March</span> 2023</span></cite></span>

</li>

<li id="cite_note-27"><span class="mw-cite-backlink"><b><a href="#cite_ref-27">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://nyu-mll.github.io/CoLA/">"The Corpus of Linguistic Acceptability (CoLA)"</a>. <i>nyu-mll.github.io</i>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201207081834/https://nyu-mll.github.io/CoLA/">Archived</a> from the original on 7 December 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">25 February</span> 2019</span>.</cite></span>

</li>

<li id="cite_note-28"><span class="mw-cite-backlink"><b><a href="#cite_ref-28">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://gluebenchmark.com/">"GLUE Benchmark"</a>. <i>gluebenchmark.com</i>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201104161928/https://gluebenchmark.com/">Archived</a> from the original on 4 November 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">25 February</span> 2019</span>.</cite></span>

</li>

<li id="cite_note-29"><span class="mw-cite-backlink"><b><a href="#cite_ref-29">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.microsoft.com/en-us/download/details.aspx?id=52398">"Microsoft Research Paraphrase Corpus"</a>. <i>Microsoft Download Center</i>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201025121243/https://www.microsoft.com/en-us/download/details.aspx?id=52398">Archived</a> from the original on 25 October 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">25 February</span> 2019</span>.</cite></span>

</li>

<li id="cite_note-30"><span class="mw-cite-backlink"><b><a href="#cite_ref-30">^</a></b></span> <span class="reference-text"><cite id="CITEREFAghaebrahimian2017" class="citation cs2">Aghaebrahimian, Ahmad (2017), "Quora Question Answer Dataset", <i>Text, Speech, and Dialogue</i>, Lecture Notes in Computer Science, vol. 10415, Springer International Publishing, pp. <span class="nowrap">66–</span>73, <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-3-319-64206-2_8">10.1007/978-3-319-64206-2_8</a>, <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9783319642055</bdi></cite></span>

</li>

<li id="cite_note-31"><span class="mw-cite-backlink"><b><a href="#cite_ref-31">^</a></b></span> <span class="reference-text"><cite id="CITEREFSammons,_V.G.Vinod_Vydiswaran,_Dan_RothVydiswaranRoth" class="citation web cs1">Sammons, V.G.Vinod Vydiswaran, Dan Roth, Mark; Vydiswaran, V.G.; Roth, Dan. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20170809113438/http://l2r.cs.uiuc.edu/~danr/Teaching/CS546-12/TeChapter.pdf">"Recognizing Textual Entailment"</a> <span class="cs1-format">(PDF)</span>. Archived from <a rel="nofollow" class="external text" href="http://l2r.cs.uiuc.edu/~danr/Teaching/CS546-12/TeChapter.pdf">the original</a> <span class="cs1-format">(PDF)</span> on 9 August 2017<span class="reference-accessdate">. Retrieved <span class="nowrap">24 February</span> 2019</span>.</cite><span class="cs1-maint citation-comment"><code class="cs1-code">{{cite web}}</code>: CS1 maint: multiple names: authors list (link)</span></span>

</li>

<li id="cite_note-32"><span class="mw-cite-backlink"><b><a href="#cite_ref-32">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://rajpurkar.github.io/SQuAD-explorer/">"The Stanford Question Answering Dataset"</a>. <i>rajpurkar.github.io</i>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201030072130/https://rajpurkar.github.io/SQuAD-explorer/">Archived</a> from the original on 30 October 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">25 February</span> 2019</span>.</cite></span>

</li>

<li id="cite_note-33"><span class="mw-cite-backlink"><b><a href="#cite_ref-33">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://nlp.stanford.edu/sentiment/treebank.html">"Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank"</a>. <i>nlp.stanford.edu</i>. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201027125825/https://nlp.stanford.edu/sentiment/treebank.html">Archived</a> from the original on 27 October 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">25 February</span> 2019</span>.</cite></span>

</li>

<li id="cite_note-34"><span class="mw-cite-backlink"><b><a href="#cite_ref-34">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://github.com/meta-llama/llama/blob/main/MODEL_CARD.md">"llama/MODEL_CARD.md at main · meta-llama/llama"</a>. <i>GitHub</i><span class="reference-accessdate">. Retrieved <span class="nowrap">28 December</span> 2024</span>.</cite></span>

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="Further_reading">Further reading</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .refbegin{margin-bottom:0.5em}.mw-parser-output .refbegin-hanging-indents>ul{margin-left:0}.mw-parser-output .refbegin-hanging-indents>ul>li{margin-left:0;padding-left:3.2em;text-indent:-3.2em}.mw-parser-output .refbegin-hanging-indents ul,.mw-parser-output .refbegin-hanging-indents ul li{list-style:none}@media(max-width:720px){.mw-parser-output .refbegin-hanging-indents>ul>li{padding-left:1.6em;text-indent:-1.6em}}.mw-parser-output .refbegin-columns{margin-top:0.3em}.mw-parser-output .refbegin-columns ul{margin-top:0}.mw-parser-output .refbegin-columns li{page-break-inside:avoid;break-inside:avoid-column}@media screen{.mw-parser-output .refbegin{font-size:90%}}

/* end https://en.wikipedia.org/ */

</style><div class="refbegin" style="">

<ul><li><cite id="CITEREFJay_M._PonteW._Bruce_Croft1998" class="citation conference cs1">Jay M. Ponte; W. Bruce Croft (1998). "A Language Modeling Approach to Information Retrieval". <i>Research and Development in Information Retrieval</i>. pp. <span class="nowrap">275–</span>281. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.4237">10.1.1.117.4237</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F290941.291008">10.1145/290941.291008</a></span>.</cite></li>

<li><cite id="CITEREFFei_SongW._Bruce_Croft1999" class="citation conference cs1">Fei Song; W. Bruce Croft (1999). "A General Language Model for Information Retrieval". <i>Research and Development in Information Retrieval</i>. pp. <span class="nowrap">279–</span>280. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.21.6467">10.1.1.21.6467</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F319950.320022">10.1145/319950.320022</a></span>.</cite></li>

<li><cite id="CITEREFChenJoshua_Goodman1998" class="citation techreport cs1">Chen, Stanley F.; Joshua Goodman (1998). <a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=273adbdb43097636aa9260d9ecd60d0787b0ef4d"><i>An Empirical Study of Smoothing Techniques for Language Modeling</i></a> (Technical report). Harvard University. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.5458">10.1.1.131.5458</a></span>.</cite></li></ul>

</div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style></div><div role="navigation" class="navbox" aria-labelledby="Natural_language_processing454" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Natural_language_processing454" style="font-size:114%;margin:0 4em"><a href="Natural_language_processing" title="Natural language processing">Natural language processing</a></div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">General terms</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="AI-complete" title="AI-complete">AI-complete</a></li>

<li><a href="Bag-of-words_model" title="Bag-of-words model">Bag-of-words</a></li>

<ul><li><a href="Bigram" title="Bigram">Bigram</a></li>

<li><a href="Trigram" title="Trigram">Trigram</a></li></ul></li>

<li><a href="Computational_linguistics" title="Computational linguistics">Computational linguistics</a></li>

<li><a href="Natural_language_understanding" title="Natural language understanding">Natural language understanding</a></li>

<li><a href="Stop_word" title="Stop word">Stop words</a></li>

<li><a href="Text_processing" title="Text processing">Text processing</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Text_mining" title="Text mining">Text analysis</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Argument_mining" title="Argument mining">Argument mining</a></li>

<li><a href="Collocation_extraction" title="Collocation extraction">Collocation extraction</a></li>

<li><a href="Concept_mining" title="Concept mining">Concept mining</a></li>

<li><a href="Coreference#Coreference_resolution" title="Coreference">Coreference resolution</a></li>

<li><a href="Deep_linguistic_processing" title="Deep linguistic processing">Deep linguistic processing</a></li>

<li><a href="Distant_reading" title="Distant reading">Distant reading</a></li>

<li><a href="Information_extraction" title="Information extraction">Information extraction</a></li>

<li><a href="Named-entity_recognition" title="Named-entity recognition">Named-entity recognition</a></li>

<li><a href="Ontology_learning" title="Ontology learning">Ontology learning</a></li>

<li><a href="Parsing" title="Parsing">Parsing</a>

<ul><li><a href="Semantic_parsing" title="Semantic parsing">Semantic parsing</a></li>

<li><a href="Syntactic_parsing_(computational_linguistics)" title="Syntactic parsing (computational linguistics)">Syntactic parsing</a></li></ul></li>

<li><a href="Part-of-speech_tagging" title="Part-of-speech tagging">Part-of-speech tagging</a></li>

<li><a href="Semantic_analysis_(machine_learning)" title="Semantic analysis (machine learning)">Semantic analysis</a></li>

<li><a href="Semantic_role_labeling" title="Semantic role labeling">Semantic role labeling</a></li>

<li><a href="Semantic_decomposition_(natural_language_processing)" title="Semantic decomposition (natural language processing)">Semantic decomposition</a></li>

<li><a href="Semantic_similarity" title="Semantic similarity">Semantic similarity</a></li>

<li><a href="Sentiment_analysis" title="Sentiment analysis">Sentiment analysis</a></li></ul>

<ul><li><a href="Terminology_extraction" title="Terminology extraction">Terminology extraction</a></li>

<li><a href="Text_mining" title="Text mining">Text mining</a></li>

<li><a href="Textual_entailment" title="Textual entailment">Textual entailment</a></li>

<li><a href="Truecasing" title="Truecasing">Truecasing</a></li>

<li><a href="Word-sense_disambiguation" title="Word-sense disambiguation">Word-sense disambiguation</a></li>

<li><a href="Word-sense_induction" title="Word-sense induction">Word-sense induction</a></li></ul>

</div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th id="Text_segmentation21" scope="row" class="navbox-group" style="width:1%"><a href="Text_segmentation" title="Text segmentation">Text segmentation</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Compound-term_processing" title="Compound-term processing">Compound-term processing</a></li>

<li><a href="Lemmatisation" class="mw-redirect" title="Lemmatisation">Lemmatisation</a></li>

<li><a href="Lexical_analysis" title="Lexical analysis">Lexical analysis</a></li>

<li><a href="Shallow_parsing" title="Shallow parsing">Text chunking</a></li>

<li><a href="Stemming" title="Stemming">Stemming</a></li>

<li><a href="Sentence_boundary_disambiguation" title="Sentence boundary disambiguation">Sentence segmentation</a></li>

<li><a href="Word#Word_boundaries" title="Word">Word segmentation</a></li></ul>

</div></td></tr></tbody></table><div>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Automatic_summarization" title="Automatic summarization">Automatic summarization</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Multi-document_summarization" title="Multi-document summarization">Multi-document summarization</a></li>

<li><a href="Sentence_extraction" title="Sentence extraction">Sentence extraction</a></li>

<li><a href="Text_simplification" title="Text simplification">Text simplification</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Machine_translation" title="Machine translation">Machine translation</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Computer-assisted_translation" title="Computer-assisted translation">Computer-assisted</a></li>

<li><a href="Example-based_machine_translation" title="Example-based machine translation">Example-based</a></li>

<li><a href="Rule-based_machine_translation" title="Rule-based machine translation">Rule-based</a></li>

<li><a href="Statistical_machine_translation" title="Statistical machine translation">Statistical</a></li>

<li><a href="Transfer-based_machine_translation" title="Transfer-based machine translation">Transfer-based</a></li>

<li><a href="Neural_machine_translation" title="Neural machine translation">Neural</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Distributional_semantics" title="Distributional semantics">Distributional semantics</a> models</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<li><a href="Document-term_matrix" title="Document-term matrix">Document-term matrix</a></li>

<li><a href="Explicit_semantic_analysis" title="Explicit semantic analysis">Explicit semantic analysis</a></li>

<li><a href="FastText" title="FastText">fastText</a></li>

<li><a href="GloVe" title="GloVe">GloVe</a></li>

<li> (<a href="Large_language_model" title="Large language model">large</a>)</li>

<li><a href="Latent_semantic_analysis" title="Latent semantic analysis">Latent semantic analysis</a></li>

<li><a href="Word_embedding" title="Word embedding">Word embedding</a></li>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Language_resource" title="Language resource">Language resources</a>,<br>datasets and corpora</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Types and<br>standards</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Corpus_linguistics" title="Corpus linguistics">Corpus linguistics</a></li>

<li><a href="Lexical_resource" title="Lexical resource">Lexical resource</a></li>

<li><a href="Linguistic_Linked_Open_Data" title="Linguistic Linked Open Data">Linguistic Linked Open Data</a></li>

<li><a href="Machine-readable_dictionary" title="Machine-readable dictionary">Machine-readable dictionary</a></li>

<li><a href="Parallel_text" title="Parallel text">Parallel text</a></li>

<li><a href="PropBank" title="PropBank">PropBank</a></li>

<li><a href="Semantic_network" title="Semantic network">Semantic network</a></li>

<li><a href="Simple_Knowledge_Organization_System" title="Simple Knowledge Organization System">Simple Knowledge Organization System</a></li>

<li><a href="Speech_corpus" title="Speech corpus">Speech corpus</a></li>

<li><a href="Text_corpus" title="Text corpus">Text corpus</a></li>

<li><a href="Thesaurus_(information_retrieval)" title="Thesaurus (information retrieval)">Thesaurus (information retrieval)</a></li>

<li><a href="Treebank" title="Treebank">Treebank</a></li>

<li><a href="Universal_Dependencies" title="Universal Dependencies">Universal Dependencies</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Data</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="BabelNet" title="BabelNet">BabelNet</a></li>

<li><a href="Bank_of_English" title="Bank of English">Bank of English</a></li>

<li><a href="DBpedia" title="DBpedia">DBpedia</a></li>

<li><a href="FrameNet" title="FrameNet">FrameNet</a></li>

<li><a href="Google_Ngram_Viewer" class="mw-redirect" title="Google Ngram Viewer">Google Ngram Viewer</a></li>

<li><a href="WordNet" title="WordNet">WordNet</a></li>

<li><a href="Wikidata" title="Wikidata">Wikidata</a></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Automatic_identification_and_data_capture" title="Automatic identification and data capture">Automatic identification<br>and data capture</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Speech_recognition" title="Speech recognition">Speech recognition</a></li>

<li><a href="Speech_segmentation" title="Speech segmentation">Speech segmentation</a></li>

<li><a href="Speech_synthesis" title="Speech synthesis">Speech synthesis</a></li>

<li><a href="Natural_language_generation" title="Natural language generation">Natural language generation</a></li>

<li><a href="Optical_character_recognition" title="Optical character recognition">Optical character recognition</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Topic_model" title="Topic model">Topic model</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Document_classification" title="Document classification">Document classification</a></li>

<li><a href="Latent_Dirichlet_allocation" title="Latent Dirichlet allocation">Latent Dirichlet allocation</a></li>

<li><a href="Pachinko_allocation" title="Pachinko allocation">Pachinko allocation</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Computer-assisted_reviewing" title="Computer-assisted reviewing">Computer-assisted<br>reviewing</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Automated_essay_scoring" title="Automated essay scoring">Automated essay scoring</a></li>

<li><a href="Concordancer" title="Concordancer">Concordancer</a></li>

<li><a href="Grammar_checker" title="Grammar checker">Grammar checker</a></li>

<li><a href="Predictive_text" title="Predictive text">Predictive text</a></li>

<li><a href="Pronunciation_assessment" title="Pronunciation assessment">Pronunciation assessment</a></li>

<li><a href="Spell_checker" title="Spell checker">Spell checker</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Natural-language_user_interface" title="Natural-language user interface">Natural language<br>user interface</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Chatbot" title="Chatbot">Chatbot</a></li>

<li><a href="Interactive_fiction" title="Interactive fiction">Interactive fiction</a></li>

<li><a href="Question_answering" title="Question answering">Question answering</a></li>

<li><a href="Virtual_assistant" title="Virtual assistant">Virtual assistant</a></li>

<li><a href="Voice_user_interface" title="Voice user interface">Voice user interface</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Related</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Formal_semantics_(natural_language)" title="Formal semantics (natural language)">Formal semantics</a></li>

<li><a href="Hallucination_(artificial_intelligence)" title="Hallucination (artificial intelligence)">Hallucination</a></li>

<li><a href="Natural_Language_Toolkit" title="Natural Language Toolkit">Natural Language Toolkit</a></li>

<li><a href="SpaCy" title="SpaCy">spaCy</a></li></ul>

</div></td></tr></tbody></table></div>

<div class="navbox-styles"></div><div role="navigation" class="navbox" aria-labelledby="Artificial_intelligence_(AI)426" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Artificial_intelligence_(AI)426" style="font-size:114%;margin:0 4em"><a href="Artificial_intelligence" title="Artificial intelligence">Artificial intelligence</a> (AI)</div></th></tr><tr><td class="navbox-abovebelow" colspan="2"><div>

<ul><li><a href="History_of_artificial_intelligence" title="History of artificial intelligence">History</a>

<ul><li><a href="Timeline_of_artificial_intelligence" title="Timeline of artificial intelligence">timeline</a></li></ul></li>

<li><a href="List_of_artificial_intelligence_companies" title="List of artificial intelligence companies">Companies</a></li>

<li><a href="List_of_artificial_intelligence_projects" title="List of artificial intelligence projects">Projects</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Concepts</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Parameter" title="Parameter">Parameter</a>

<ul><li><a href="Hyperparameter_(machine_learning)" title="Hyperparameter (machine learning)">Hyperparameter</a></li></ul></li>

<li><a href="Loss_functions_for_classification" title="Loss functions for classification">Loss functions</a></li>

<li><a href="Regression_analysis" title="Regression analysis">Regression</a>

<ul><li><a href="Bias%E2%80%93variance_tradeoff" title="Bias–variance tradeoff">Bias–variance tradeoff</a></li>

<li><a href="Double_descent" title="Double descent">Double descent</a></li>

<li><a href="Overfitting" title="Overfitting">Overfitting</a></li></ul></li>

<li><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></li>

<li><a href="Gradient_descent" title="Gradient descent">Gradient descent</a>

<li><a href="Quasi-Newton_method" title="Quasi-Newton method">Quasi-Newton method</a></li>

<li><a href="Conjugate_gradient_method" title="Conjugate gradient method">Conjugate gradient method</a></li></ul></li>

<li><a href="Backpropagation" title="Backpropagation">Backpropagation</a></li>

<li><a href="Attention_(machine_learning)" title="Attention (machine learning)">Attention</a></li>

<li><a href="Convolution" title="Convolution">Convolution</a></li>

<li><a href="Normalization_(machine_learning)" title="Normalization (machine learning)">Normalization</a>

<ul><li><a href="Batch_normalization" title="Batch normalization">Batchnorm</a></li></ul></li>

<li><a href="Activation_function" title="Activation function">Activation</a>

<ul><li><a href="Softmax_function" title="Softmax function">Softmax</a></li>

<li><a href="Sigmoid_function" title="Sigmoid function">Sigmoid</a></li>

<li><a href="Rectifier_(neural_networks)" title="Rectifier (neural networks)">Rectifier</a></li></ul></li>

<li><a href="Gating_mechanism" title="Gating mechanism">Gating</a></li>

<li><a href="Weight_initialization" title="Weight initialization">Weight initialization</a></li>

<li><a href="Regularization_(mathematics)" title="Regularization (mathematics)">Regularization</a></li>

<li><a href="Training%2C_validation%2C_and_test_data_sets" title="Training, validation, and test data sets">Datasets</a>

<ul><li><a href="Data_augmentation" title="Data augmentation">Augmentation</a></li></ul></li>

<li><a href="Prompt_engineering" title="Prompt engineering">Prompt engineering</a></li>

<li><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a>

<ul><li><a href="Q-learning" title="Q-learning">Q-learning</a></li>

<li><a href="State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action" title="State–action–reward–state–action">SARSA</a></li>

<li><a href="Imitation_learning" title="Imitation learning">Imitation</a></li>

<li><a href="Policy_gradient_method" title="Policy gradient method">Policy gradient</a></li></ul></li>

<li><a href="Diffusion_process" title="Diffusion process">Diffusion</a></li>

<li><a href="Latent_diffusion_model" title="Latent diffusion model">Latent diffusion model</a></li>

<li><a href="Autoregressive_model" title="Autoregressive model">Autoregression</a></li>

<li><a href="Adversarial_machine_learning" title="Adversarial machine learning">Adversary</a></li>

<li><a href="Uncanny_valley" title="Uncanny valley">Uncanny valley</a></li>

<li><a href="Self-supervised_learning" title="Self-supervised learning">Self-supervised learning</a></li>

<li><a href="Reflection_(artificial_intelligence)" class="mw-redirect" title="Reflection (artificial intelligence)">Reflection</a></li>

<li><a href="Recursive_self-improvement" title="Recursive self-improvement">Recursive self-improvement</a></li>

<li><a href="Hallucination_(artificial_intelligence)" title="Hallucination (artificial intelligence)">Hallucination</a></li>

<li><a href="Word_embedding" title="Word embedding">Word embedding</a></li>

<li><a href="Vibe_coding" title="Vibe coding">Vibe coding</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Applications</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Machine_learning" title="Machine learning">Machine learning</a>

<ul><li><a href="Prompt_engineering#In-context_learning" title="Prompt engineering">In-context learning</a></li></ul></li>

<li><a href="Neural_network_(machine_learning)" title="Neural network (machine learning)">Artificial neural network</a>

<ul><li><a href="Deep_learning" title="Deep learning">Deep learning</a></li></ul></li>

<li>

<ul><li><a href="Large_language_model" title="Large language model">Large language model</a></li>

<li><a href="Reasoning_language_model" title="Reasoning language model">Reasoning language model</a></li>

<li><a href="Model_Context_Protocol" title="Model Context Protocol">Model Context Protocol</a></li>

<li><a href="Intelligent_agent" title="Intelligent agent">Intelligent agent</a></li>

<li><a href="Artificial_human_companion" title="Artificial human companion">Artificial human companion</a></li>

<li><a href="Humanity's_Last_Exam" title="Humanity's Last Exam">Humanity's Last Exam</a></li>

<li><a href="Artificial_general_intelligence" title="Artificial general intelligence">Artificial general intelligence (AGI)</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Implementations</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Audio–visual</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="AlexNet" title="AlexNet">AlexNet</a></li>

<li><a href="WaveNet" title="WaveNet">WaveNet</a></li>

<li><a href="Human_image_synthesis" title="Human image synthesis">Human image synthesis</a></li>

<li><a href="Computer_vision" title="Computer vision">Computer vision</a></li>

<li><a href="Deep_learning_speech_synthesis" title="Deep learning speech synthesis">Speech synthesis</a>

<li><a href="ElevenLabs" title="ElevenLabs">ElevenLabs</a></li></ul></li>

<li><a href="Speech_recognition" title="Speech recognition">Speech recognition</a>

<ul><li><a href="Whisper_(speech_recognition_system)" title="Whisper (speech recognition system)">Whisper</a></li></ul></li>

<li><a href="Facial_recognition_system" title="Facial recognition system">Facial recognition</a></li>

<li><a href="AlphaFold" title="AlphaFold">AlphaFold</a></li>

<li><a href="Text-to-image_model" title="Text-to-image model">Text-to-image models</a>

<ul><li><a href="Aurora_(text-to-image_model)" class="mw-redirect" title="Aurora (text-to-image model)">Aurora</a></li>

<li><a href="Adobe_Firefly" title="Adobe Firefly">Firefly</a></li>

<li><a href="Ideogram_(text-to-image_model)" title="Ideogram (text-to-image model)">Ideogram</a></li>

<li><a href="Imagen_(text-to-image_model)" title="Imagen (text-to-image model)">Imagen</a></li>

<li><a href="Midjourney" title="Midjourney">Midjourney</a></li>

<li><a href="Recraft" title="Recraft">Recraft</a></li>

<li><a href="Stable_Diffusion" title="Stable Diffusion">Stable Diffusion</a></li></ul></li>

<li><a href="Text-to-video_model" title="Text-to-video model">Text-to-video models</a>

<ul><li><a href="Dream_Machine_(text-to-video_model)" title="Dream Machine (text-to-video model)">Dream Machine</a></li>

<li><a href="Runway_(company)#Services_and_technologies" title="Runway (company)">Runway Gen</a></li>

<li><a href="MiniMax_(company)#Hailuo_AI" title="MiniMax (company)">Hailuo AI</a></li>

<li><a href="Kling_(text-to-video_model)" class="mw-redirect" title="Kling (text-to-video model)">Kling</a></li>

<li><a href="Music_and_artificial_intelligence" title="Music and artificial intelligence">Music generation</a>

<ul><li><a href="Riffusion" title="Riffusion">Riffusion</a></li>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Text</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<li><a href="GloVe" title="GloVe">GloVe</a></li>

<li><a href="Llama_(language_model)" title="Llama (language model)">Llama</a></li>

<li><a href="Chinchilla_(language_model)" title="Chinchilla (language model)">Chinchilla AI</a></li>

<li><a href="ChatGPT" title="ChatGPT">ChatGPT</a></li>

<li><a href="Claude_(language_model)" title="Claude (language model)">Claude</a></li>

<li><a href="Gemini_(language_model)" title="Gemini (language model)">Gemini</a>

<ul><li><a href="Gemini_(chatbot)" title="Gemini (chatbot)">chatbot</a></li></ul></li>

<li><a href="LaMDA" title="LaMDA">LaMDA</a></li>

<li><a href="BLOOM_(language_model)" title="BLOOM (language model)">BLOOM</a></li>

<li><a href="Project_Debater" title="Project Debater">Project Debater</a></li>

<li><a href="IBM_Watson" title="IBM Watson">IBM Watson</a></li>

<li><a href="IBM_Watsonx" title="IBM Watsonx">IBM Watsonx</a></li>

<li><a href="IBM_Granite" title="IBM Granite">Granite</a></li>

<li><a href="Huawei_PanGu" title="Huawei PanGu">PanGu-Σ</a></li>

<li><a href="DeepSeek_(chatbot)" title="DeepSeek (chatbot)">DeepSeek</a></li>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Decisional</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="AlphaGo" title="AlphaGo">AlphaGo</a></li>

<li><a href="AlphaZero" title="AlphaZero">AlphaZero</a></li>

<li><a href="OpenAI_Five" title="OpenAI Five">OpenAI Five</a></li>

<li><a href="Self-driving_car" title="Self-driving car">Self-driving car</a></li>

<li><a href="MuZero" title="MuZero">MuZero</a></li>

<li><a href="Action_selection" title="Action selection">Action selection</a>

<ul><li><a href="AutoGPT" title="AutoGPT">AutoGPT</a></li></ul></li>

<li><a href="Robot_control" title="Robot control">Robot control</a></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">People</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Alan_Turing" title="Alan Turing">Alan Turing</a></li>

<li><a href="Warren_Sturgis_McCulloch" title="Warren Sturgis McCulloch">Warren Sturgis McCulloch</a></li>

<li><a href="Walter_Pitts" title="Walter Pitts">Walter Pitts</a></li>

<li><a href="John_von_Neumann" title="John von Neumann">John von Neumann</a></li>

<li><a href="Claude_Shannon" title="Claude Shannon">Claude Shannon</a></li>

<li><a href="Shun'ichi_Amari" title="Shun'ichi Amari">Shun'ichi Amari</a></li>

<li><a href="Kunihiko_Fukushima" title="Kunihiko Fukushima">Kunihiko Fukushima</a></li>

<li><a href="Takeo_Kanade" title="Takeo Kanade">Takeo Kanade</a></li>

<li><a href="Marvin_Minsky" title="Marvin Minsky">Marvin Minsky</a></li>

<li><a href="John_McCarthy_(computer_scientist)" title="John McCarthy (computer scientist)">John McCarthy</a></li>

<li><a href="Nathaniel_Rochester_(computer_scientist)" title="Nathaniel Rochester (computer scientist)">Nathaniel Rochester</a></li>

<li><a href="Allen_Newell" title="Allen Newell">Allen Newell</a></li>

<li><a href="Cliff_Shaw" title="Cliff Shaw">Cliff Shaw</a></li>

<li><a href="Herbert_A._Simon" title="Herbert A. Simon">Herbert A. Simon</a></li>

<li><a href="Oliver_Selfridge" title="Oliver Selfridge">Oliver Selfridge</a></li>

<li><a href="Frank_Rosenblatt" title="Frank Rosenblatt">Frank Rosenblatt</a></li>

<li><a href="Bernard_Widrow" title="Bernard Widrow">Bernard Widrow</a></li>

<li><a href="Joseph_Weizenbaum" title="Joseph Weizenbaum">Joseph Weizenbaum</a></li>

<li><a href="Seymour_Papert" title="Seymour Papert">Seymour Papert</a></li>

<li><a href="Seppo_Linnainmaa" title="Seppo Linnainmaa">Seppo Linnainmaa</a></li>

<li><a href="Paul_Werbos" title="Paul Werbos">Paul Werbos</a></li>

<li><a href="Geoffrey_Hinton" title="Geoffrey Hinton">Geoffrey Hinton</a></li>

<li><a href="John_Hopfield" title="John Hopfield">John Hopfield</a></li>

<li><a href="J%C3%BCrgen_Schmidhuber" title="Jürgen Schmidhuber">Jürgen Schmidhuber</a></li>

<li><a href="Yann_LeCun" title="Yann LeCun">Yann LeCun</a></li>

<li><a href="Yoshua_Bengio" title="Yoshua Bengio">Yoshua Bengio</a></li>

<li><a href="Lotfi_A._Zadeh" title="Lotfi A. Zadeh">Lotfi A. Zadeh</a></li>

<li><a href="Stephen_Grossberg" title="Stephen Grossberg">Stephen Grossberg</a></li>

<li><a href="Alex_Graves_(computer_scientist)" title="Alex Graves (computer scientist)">Alex Graves</a></li>

<li><a href="James_Goodnight" title="James Goodnight">James Goodnight</a></li>

<li><a href="Andrew_Ng" title="Andrew Ng">Andrew Ng</a></li>

<li><a href="Ilya_Sutskever" title="Ilya Sutskever">Ilya Sutskever</a></li>

<li><a href="Alex_Krizhevsky" title="Alex Krizhevsky">Alex Krizhevsky</a></li>

<li><a href="Ian_Goodfellow" title="Ian Goodfellow">Ian Goodfellow</a></li>

<li><a href="Demis_Hassabis" title="Demis Hassabis">Demis Hassabis</a></li>

<li><a href="David_Silver_(computer_scientist)" title="David Silver (computer scientist)">David Silver</a></li>

<li><a href="Andrej_Karpathy" title="Andrej Karpathy">Andrej Karpathy</a></li>

<li><a href="Ashish_Vaswani" title="Ashish Vaswani">Ashish Vaswani</a></li>

<li><a href="Noam_Shazeer" title="Noam Shazeer">Noam Shazeer</a></li>

<li><a href="Aidan_Gomez" title="Aidan Gomez">Aidan Gomez</a></li>

<li><a href="Mustafa_Suleyman" title="Mustafa Suleyman">Mustafa Suleyman</a></li>

<li><a href="Fran%C3%A7ois_Chollet" title="François Chollet">François Chollet</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Architectures</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Neural_Turing_machine" title="Neural Turing machine">Neural Turing machine</a></li>

<li><a href="Differentiable_neural_computer" title="Differentiable neural computer">Differentiable neural computer</a></li>

<li><a href="Transformer_(deep_learning_architecture)" title="Transformer (deep learning architecture)">Transformer</a>

<ul><li><a href="Vision_transformer" title="Vision transformer">Vision transformer (ViT)</a></li></ul></li>

<li><a href="Recurrent_neural_network" title="Recurrent neural network">Recurrent neural network (RNN)</a></li>

<li><a href="Long_short-term_memory" title="Long short-term memory">Long short-term memory (LSTM)</a></li>

<li><a href="Gated_recurrent_unit" title="Gated recurrent unit">Gated recurrent unit (GRU)</a></li>

<li><a href="Echo_state_network" title="Echo state network">Echo state network</a></li>

<li><a href="Multilayer_perceptron" title="Multilayer perceptron">Multilayer perceptron (MLP)</a></li>

<li><a href="Convolutional_neural_network" title="Convolutional neural network">Convolutional neural network (CNN)</a></li>

<li><a href="Residual_neural_network" title="Residual neural network">Residual neural network (RNN)</a></li>

<li><a href="Highway_network" title="Highway network">Highway network</a></li>

<li><a href="Mamba_(deep_learning_architecture)" title="Mamba (deep learning architecture)">Mamba</a></li>

<li><a href="Autoencoder" title="Autoencoder">Autoencoder</a></li>

<li><a href="Variational_autoencoder" title="Variational autoencoder">Variational autoencoder (VAE)</a></li>

<li><a href="Generative_adversarial_network" title="Generative adversarial network">Generative adversarial network (GAN)</a></li>

<li><a href="Graph_neural_network" title="Graph neural network">Graph neural network (GNN)</a></li></ul>

</div></td></tr><tr><td class="navbox-abovebelow" colspan="2"><div>

<ul><li><span class="noviewer" typeof="mw:File"><span title="Category"></span></span> Category</li></ul>

</div></td></tr></tbody></table></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-07-30" href="https://en.wikipedia.org/wiki/?title=Language_model&oldid=1303340535">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>